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KEY SEGMENT SPOTTING IN VOICE MESSAGES 



Related Applications 

The present application is related to U.S. Patent 

Application Serial No. (Attorney Docket No. Lee 23-2), 

entitled VOICE MESSAGE FILTERING FOR CLASSIFICATION OF VOICE 
5 MESSAGE ACCORDING TO CALLER, filed on even date herewith and 
incorporated herein by reference in its entirety. 

Field Of The Invention 

The present invention relates to voice messaging 
10 systems and methods, in particular, key segment spotting in voice 
messages . 

Background Information 

In voice messaging (or "'voice-mail") systems, a user is 
often forced to listen to multiple, often lengthy messages to 
15 obtain certain items of essential information such as the names 

of the callers who have left the messages and the callers' return 
telephone numbers. This can be a tedious and time-consuming 
process. Furthermore, the manual process of transcribing the 
essential information is susceptible to errors. 

20 Summary Of The Invention 

The present invention is directed to a method and 
system of identifying and spotting segments containing key 
information in voice messages. For example, the method of the 
present invention can be used to spot a name segment in a voice 

25 message by detecting and verifying the presence of a segment such 
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as ''My name is ..." or ''This is The method can also be 

used to spot a phone number segment by detecting and verifying 
the presence of a segment such as "My number is ..." or "Call me 
back at ..." or by spotting the numerical part of the message 
5 such as "[my number is] 3-6-4-7-5-8-9". Once the key segment of 
interest has been spotted, the method or system of the present 
invention can provide the user with only the pertinent 
information (e.g., the name of the caller) contained in the key 
segment. The method of the present invention can spot the key 
10 segments and can then retrieve only the desired segments. This 

allows a user retrieving a message to hear just a desired section 
or sections of a message without having to listen to the rest of 
the message. 

The method of the present invention is advantageously 
15 useful in sorting through a large number of voice mail messages. 
The method speeds up the process of searching for particular 
messages, messages from particular callers, or for certain 
segments within messages. 

Brief Description Of The Drawing 
20 FIG. 1 illustrates a key segment registration procedure 

in accordance with the present invention. 

FIG. 2 illustrates the handling of voice messages in 
accordance with the present invention. 

FIG. 3 illustrates the retrieval of key segments and 
25 messages with key segments, in accordance with the present 
invention. 

Detailed Description 

In an exemplary embodiment of a method in accordance 
with the present invention, key segment spotting is achieved by 
30 first having a user register the key segments he would like to 
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spot in the messages. This procedure is illustrated in FIG. 1. 
As shown, the registration of key segments can be done by text 
input (e.g., if a keyboard is available;, the user can type in the 
key segment to be registered) or by voice input (e.g., the user 
5 speaks the key segment to be registered) . 

Also, the user may register a key segment by using part 
of an actual voice message. As shown in FIG. 1, a user, while 
playing back a stored voice message, can mark at 13 a key segment 
within the message, by pressing, for example, the ^'B" key to mark 

10 the beginning of the key segment and the ^^E" key to mark the end 
of the key segment. By pressing a further key sequence, e.g., 
^^S, the user can indicate that the marked segment, delimited 
with the B and E key presses, is to be registered. This feature 
is useful, for example, for saving the names of the message 

15 sender as spoken by the senders in order to spot them later. 

Commonly occurring key segments such as name segments, 
phone number segments and date segments may be provided without 
registration as predefined segments. As discussed below, such 
predefined segments can be retrieved by pressing predefined key 

20 sequences . 

As shown in FIG, 1, the user can input a key segment to 
be registered either as text, speech, pronunciation or by marking 
a segment within a message. Text can be entered, for example, 
with an alphanumeric key pad (not shown) , keyboard or any other 

25 such text-entry device. A speech representation of a key segment 
can be entered, for example, via the audio path of a telephone 
(such as the user might use to dial into the system of the 
present invention.) The pronunciation can be specified using any 
set of symbols, such as the IPA symbol set. The symbols can be 

30 entered, for example, as text. 
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If entered as text, the text of the key segment is 
processed at 11 through a text-to-speech front end to obtain the 
pronunciation of the key segment. For example, if the user 
enters the word ''four", the text-to-speech front end would 
5 generate the IPA symbol sequence f-ow-r to represent the 

pronunciation. If the user speaks the key segment or marks the 
key segment in a message, the key segment is processed at 12 to 
generate its pronunciation using speech recognition. 

An identifier of the key segment (e.g., a segment name) 

10 and the corresponding characteristics (e.g., the pronunciation) 
of the key segment are stored at 15 in a storage device or 
memory. The text-to-speech and speech recognition functions can 
be implemented in conventional ways using known methods and 
systems. For example, the speech recognition function 12 can be 

15 implemented in accordance with the methods and systems described 
in U.S. Patents Nos. 4,713,777, 4,718,088, 5,509,104, 5,579,436, 
and/or 5,64 9,057. The text-to-speech function can be implemented 
as described in "Multilingual Text-to-Speech Synthesis: The Bell 
Labs Approach," by R. W. Sproat, Kluwer Academic Publishers, 

20 1998. 

As voice messages are received, the messages are 
processed as illustrated in FIG. 2 in order to search for 
registered and/or predefined key segments. Using the key segment 
characteristics stored at 15 and speaker-independent models (for 

25 the sound units of the pronunciation) key segment detection is 

performed at 21 to spot one or more registered or predefined key 
segments in a voice message. The key segment detection at 21 can 
be implemented in a known way using conventional wordspotting or 
phrase detection technology, such as described in U.S. Patent No. 

30 5,509,104. 
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To enhance the accuracy of the key segment detection, 
utterance verification is performed at 23 on the key segments 
detected at 21. Utterance verification is used to confirm that 
the segments detected at 21 contain the information that is 
5 sought. Utterance verification can be performed as described, 

for example, in U.S. Patent No. 5,675,706. The messages are then 
tagged at 25 with the key segments and the locations of the key 
segments in the messages to facilitate their later retrieval. In 
one exemplary embodiment, each message is stored with a header 

10 containing tag information. The tag information, for example, 
may indicate the locations of key segments detected within the 
message. The location of each key segment can be represented, 
for example, as an offset in time or address space from the 
beginning of the message. 

15 Messages in which no registered or predefined key 

segments are detected can be stored in a conventional manner 
without being tagged and can be retrieved in a conventional 
manner . 

Once one or more messages have been tagged and stored, 
20 the messages and/or key segments within the messages can be 
retrieved. An exemplary message retrieval procedure in 
accordance with the present invention is illustrated in FIG. 3. 

The retrieval procedure is initiated when a user enters 
an enquiry for a key segment. The enquiry can be entered by a 
25 variety of means, including speech (i.e., speaking the desired 
key segment), by typing the name or pronunciation of the key 
segment, or by pressing a sequence of one or more buttons on a 
keypad, wherein the sequence identifies the desired key segment. 

Upon receiving the user enquiry for a key segment, the 
30 procedure first determines at 31 whether the user has entered the 
enquiry by speech, i.e., if the user has spoken the name of the 
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key segment. If so^ operation proceeds to 33 in which speech 
recognition is performed on the spoken enquiry to determine the 
segment name spoken. 

Operation then proceeds to 35 in which it is determined 
5 if the specified key segment is one that has been predefined or 
already registered. If the key segment to which the user's 
enquiry pertains is registered or is one of the predefined 
segments, operation proceeds to 37 in which a search for the 
specified key segment is performed in the tagged messages. At 
10 39, the specified key segment is retrieved from those messages in 
which it was found. If the enquired-about key segment is found 
in multiple messages, each occurrence of the key segment is 
retrieved. 

To access predefined key segments, the user may press 
15 predefined key sequences on the user's telephone dial pad, such 
as, for the telephone number segment, for the name 

segment, ^^D for the date segment, and so on. Furthermore, 
telephone number detection with can include number 

verification. A number retrieved from a segment of a message can 
20 optionally be dialed by pressing a predefined key sequence (e.g.. 

If it is determined at 35 that the key segment to which 
the user's enquiry pertains is a new key segment (i.e., it is not 
a predefined or registered segment), the characteristics (e.g., 

25 pronunciation) of the key segment are first obtained at 36 with 
the procedure of FIG. 1. The stored messages are then tagged at 
38, as per the message handler procedure of FIG. 2, to indicate 
where, if at all, the newly specified key segment is found in the 
stored messages. Once the messages have been tagged with respect 

30 to the new key segment, the key segment is retrieved at 39, as 
described above. 
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When a key segment is retrieved at 39 from a message, 
the user can opt to save the retrieved key segment for future use 
as a key segment by pressing a predefined sequence of keys (e.g., 
^*S) . Furthermore, if a name segment is retrieved, it can be 
5 used to identify the caller and hence can be used for message 

filtering and classification of messages according to the caller. 
This enables the system of the present invention to save for 
example, the message sender's name in their own voice for later 
use in identifying, tagging and retrieving the sender's messages. 

10 The present invention uses speech recognition, 

wordspotting, key-word detection and utterance verification 
technologies for spotting key segments in messages. It can also 
use speech coding technology for key segment spotting in coded 
voice mail messages. 

15 The present invention can be implemented as part of a 

voice messaging system, such as the AUDIX system, available from 
Lucent Technologies, Inc. The present invention can be 
implemented on a general purpose computer with software or with 
special purpose hardware. 



7 

[NYC] 253300.1 



What is claimed is: 



1. A method of spotting a key segment in a voice message 
comprising the steps of: 

identifying a key segment; 
receiving a voice message; 

detecting the key segment in the voice message; 
tagging the voice message so as to indicate the 
location of the detected key segment within the voice message; 
receiving an enquiry for the key segment; and 
retrieving the key segment from the voice message, 

2. The method of claim 1, wherein the step of identifying a key 
segment includes registering the key segment by storing an 
identification and a characteristic of the key segment. 

3. The method of claim 1, wherein the step of identifying a key 
segment includes predefining the key segment. 

4. The method of claim 1, wherein the enquiry for the key 
segment includes speech. 

5. The method of claim 2, wherein the characteristic of the key 
segment includes a pronunciation of the key segment. 

6. A method of spotting a key segment in a voice message 
comprising the steps of: 

receiving a voice message; 
receiving an enquiry for a key segment; 
detecting the key segment in the voice message; 
tagging the voice message with the location of the 
detected key segment; and 
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retrieving the key segment from the voice message. 

7. The method of claim 6, comprising the step of registering 
the key segment by storing an identification and a characteristic 
of the key segment. 

8. The method of claim 1, wherein the characteristic of the key 
segment includes a pronunciation of the key segment 
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Abstract Of The Disclosure 



A method and system of identifying and spotting 
segments containing key information in voice messages. The 
method can be used to spot a key segment such as a name segment 
5 in a voice message by detecting and verifying the presence of a 
phrase such as ""My name is ..." or ''This is Once the key 

segment of interest has been spotted, the method provides the 
user with only the pertinent information (e.g., the name of the 
caller), which is contained in the key segment. This allows a 
10 user retrieving a message to hear just a desired section or 
sections of a message without listening to the rest of the 
message . 
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IN THE UNITED STATES 
PATENT AND TRADEMARK OFFICE 

Declaration and Power of Attorney 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name. 

I believe I am an original, first and joint inventor of the subject matter which is claimed 
and for which a patent is sought on the invention entitled KEY SEGEMENT SPOTTING IN 
VOICE MESSAGES the specification of which is filed herewith. 

I hereby state that I have reviewed and understand the contents of the above-identified 
specification, mcluding the claims, as amended by an amendment, if any, specifically refened to 
in this oath or declaration. 

I acknowledge the duty to disclose all information known to me which is material to 
patentability as defined in Title 37, Code of Federal Regulatioxis, 1 .56, 

I hereby claim foreign priority benefits under Title 35^ United States Code, U9 of any 
foreign application(s) for patent or inventor's certificate listed below and have also identified 
below any foreign application for patent or inventor's certificate having a filing date before that 
of the application on which priority is claimed: 

- None - 

I hereby claim the benefit under Title 35, United States Code, 120 of any United States 
application(s) listed below and, insofar as the subject matter of each of the claims of this 
application is not disclosed in the prior United States application in the manner provided by the 
first paragraph of Title 35, United States Code, 112, 1 acknowledge the duty to disclose all 
information known to me to be material to patentability as defined in Title 37, Code of Federal 
Regulations, 1 ,56 which became available between the filing date of the prior application and the 
national or PCT international filing date of this application: 

- None - 

I hereby declare that all statements made herein of ray own knowledge are true and that 
all statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willfiil false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States 
Code and that such willfiil false statements may jeopardize the validity of the application or any 
patent issued thereon. 
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I hereby appoint the following attomey(s) with ftill power of substitution and revocation, 
to prosecute said application, to make alterations and amendments fhereiu, to receive the patent^ 
and to transact all business in the Patent and Trademark Office connected therewith: 

Lester H. Bimbaum (Reg. No. 25830) 

Richard J. Botos (Reg. No. 3201 6) 

Jeffery J. Brosemer (Reg. No. 36096) 

Kenneth M. Brown (Reg. No. 37590) 

Donald P. Dmella (Reg. No. 39961) 

Guy Eriksen (Reg. No. P-41736) 

Martin I. Fmston (Reg. No. 31613) 

James H. Fox (Reg. No. 29379) 

Wiiliara S. Francos (Reg. No. 3 8456) 

Bany H. Freedman (Reg. No. 261 66) 

Julio A. Garceran (Reg. No. 37138) 

Mony R. Ghose (Reg. No. 38 159) 

Jumray Goo (Reg. No. 36528) 

Anthony Grillo (Reg. No. 36535) 

Stephen M. Gurey (Reg. No. 27336) 

John M. Harman (Reg. No. 38 1 73) 

John W. Hayes (Reg. No. 33900) 

Mark A. Kurisko (Reg. No. 38944) 

Irena Lager (Reg. No. 39260) 

Christopher N. Malvone (Reg. No. 34866) 

Scott W. McLellan (Reg. No. 30776) 

Martin G. Meder (Reg. No. 34674) 

Geraldine Monteleone (Reg. No. 40097) 

John C. Moran (Reg. No. 30782) 

Michael A. Morra (Reg. No. 28975) 

Gregory J. Murgia (Reg, No. 41 209) 

Claude R. Narcisse (Reg. No. 38979) 

Joseph LOpalach (Reg. No. 36229) 

Neil R, Ormos (Reg, No. 35309) 

Eugen E. Pacher (Reg. No. 29964) 

Jack R. Penrod (Reg. No. 3 1 864) 

Daniel J. Piotrowski (Reg. No. P-42079) 

Gregory CRanieri (Reg. No. 29695) 

Scott J. Rittman (Reg. No. 3901 0) 

Eugene J. Rosenthal (Reg. No. 36658) 

Bruce S. Schneider (Reg. No. 27949) 

Ronald D. Slusky (Reg. No. 26585) 

David L. Smith (Reg. No. 30592) 

Patricia A. Verlangieri (Reg. No. P-42201) 

JohnP Veschi (Reg. No. 39058) 

David Volejnicek (Reg. No. 29355) 

Charles L. Warren (Reg. No. 27407) 

Eli Weiss (Reg. No. 17765) 
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I hereby appoint the attorneys listed on ATTACHMENT A as associate attorneys in the 
aforementioned application, with full power solely to prosecute said application, to make 
alterations and amendments therein, to receive the patent and to transact all business in the Patent 
and Trademark Office connected Avith the prosecution of said application. No other powers are 
granted to such associate attorneys and such associate attorneys are specifically denied any 
power of substitution or revocation. 



Full name of first inventor: Chin-Hui Lee 



Liventor's signature: 




Date: 




Residence: Basking Ridge, Somerset County, New Jersey 



Citizenship: United States 



Post Office Address: 44 Pheasant Run Drive 

Basking Ridge, N.3. 09720 
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Full name of second inventor:Padma Raxnesh 



inventor's signature: VcJL^ (p^ .^yjU Date: C^^/)')'^^ 

Residence: New Providence, Union County, New Jersey 
Citizenship: India 



Post Office Address: 70 Primrose Drive 

New Providence, N J. 07974 
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ATTACHMENT A 

Chris Kolefas (Reg. No. 35226) 

Telephone calls should be made to BAKER & McKENZIE at: 
Telephone No,: 212-891-3956 
Fax No.: 212-759-9133 

Please address all correspondence to: 

BAKER & McKENZIE 
805 Third Avenue 
New York, N.Y. 10022 
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